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>: 

! We investigate the algebraic complexity of tensor calulus. We consider 

QO ■ a generalization of iterated matrix product to tensors and show that the 

resulting formulas exactly capture VP, the class of polynomial families 

■ efficiently computable by arithmetic circuits. This gives a natural and 
iT^ robust characterization of this complexity class that despite its naturalness 

. is not very well understood so far. 
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1. Introduction 



The question of which polynomials can be computed by polynomial size arithmetic 
circuits is one of the central questions of algebraic complexity. It was first brought up 
explicitly by Valiant |10j who formulated a complexity theory in this setting with its 
own complexity classes and notions of completeness. Efficient computation in Valiant's 
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model is formalized by the complexity class VP which consists of families of polyno- 
mials that can be computed by arithmetic circuits of polynomial size. Despite recent 
efforts relating VP to logically defined classes of polynomial families [5J [3] , this class 
is not very well understood. This is reflected in the low number of helpful alternative 
characterizations and the conspicuous absence of any known natural complete problem. 

Consequently, most progress in arithmetic circuit complexity has not been achieved 
by considering arithmetic circuits directly, but instead by considering the somewhat 
more restricted model of arithmetic branching programs (see e.g. [5J [TUl HI H] ) • Arith- 
metic branching programs are widely conjectured to have expressivity strictly between 
that of arithmetic formulas and circuits, but have so far been better to handle with 
known proof techniques. One of the nice properties of branching programs that has of- 
ten played a crucial role is that they can equivalently be seen as computing a specified 
entry of the iterated product of a polynomial number of matrices. 

We extend this view on branching programs by going from matrices to higher dimen- 
sional tensors. Consequently, we also go from matrix product to the the generalized 
notion of contraction of tensors. It turns out that generalizing iterated matrix product 
to iterated tensor contractions does increase the expressive power of the model and 
that the resulting tensor formulas capture exactly VP. This characterization of VP 
turns out to be fairly robust in the sense that one can add different restrictions on the 
dimensions of the tensors without changing the expressive power of the model at all. 

This is not the first time that the complexity of tensor calculus is studied. Damm, 
Holzer and McKenzie[2] have characterized different boolean complexity classes by 
formulas having matrices as inputs and using addition, matrix product and tensor 
product as operations. Malod [5] adapted these formulas to the arithmetic circuit 
setting and showed characterizations for most arithmetic circuit classes. One difference 
between these results and those in our paper is that in [2] and [5] tensors are always 
encoded as matrices, i.e. the tensor product is expressed as the Kronecker product of 
two matrices. Another difference is that both the characterization of VP obtained in 
[5] and the similar characterization of LOGCFL (the Boolean analogon of VP) from 
[5] require an additional restriction, called tameness on the size of matrices computed 
at each gate of the formula. This restriction permits to control the growth of the 
intermediate objects in the computation but may seem not very natural. In this 
present work, working directly with tensors instead of a matrix representation makes 
such a unnecessary and a more direct connection between VP and tensor calculus is 
established. 

The paper is organized in three parts: We first give the definitions and properties 
of the objects we will deal with necessary to understand the remainder of the paper. 
In the second part we prove the lower bound of our characterization, i.e. we show 
our tensor formulas can efficiently simulate arithmetic circuits. In the third part 
we prove the corresponding upper bound: We first study a very restrictive class of 
tensor formulas and show that polynomials computed by them can also be computed 
by arithmetic circuits. Then we extend this result in several steps to general tensor 
formulas of polynomial size. 
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2. Preliminaries 

In the following, K is a field and bold letters denote tuples when there is no ambiguity 
on their length. 

2.1. Arithmetic circuits 

We will use the well known model of arithmetic circuits to measure the complexity 
of polynomials. In this section we give some definitions and well known properties of 
arithmetic circuits. For more background see e.g. [TJIS]- 

An arithmetic circuit is a directed acyclic graph with vertices of indegree or 2 called 
gates. The gates of indegree are called the inputs and are labeled with elements of 
K or variables. The gates of indegree 2, called computation gates, are labeled with 
operations of the field (+ and x). The polynomial computed by a gate is defined 
inductively. The polynomial computed by an input gate is the one corresponding to 
its label. The polynomial computed by a computation gate is the sum or the product of 
the polynomials computed by its children. We assume that there exists a distinguished 
gate called the output. The polynomial computed by an arithmetic circuit is the one 
computed by its output gate. The size of a circuit C, denoted by |C|, is the number 
of vertices of its underlying DAG. 

An arithmetic circuit C is said to be skew if for each x-gate at least one of its 
children is an input of the circuit. A circuit is said to be multiplicatively disjoint if for 
each x-gate, its two input subcircuits are disjoint. 

A family (f n )neH of polynomials is in VP if there exists a family of multiplicatively 
disjoint circuits (C n ) ne N and a polynomial P such that for all n € N, C n computes /„ 
and \C n \ < P(n). The family (/„) is in VP WS if the C n are skew. 

Remark 1. Originally, VP was defined as families of polynomials that can be computed 
by polynomial size circuits and have polynomially bounded degree. As shown in Jfjjl 
the definition given here is equivalent to the original one. We prefer this one here 
because the semantic condition on the degree is harder to deal with than multiplicatively 
disjointness which is more syntactic. 

In the following, we will simulate arithmetic circuits by formulas computing tensors. 
We use the notion of parse trees of a circuit. For a multiplicatively disjoint circuit C , 
we define its parse trees inductively. A parse tree T of C is a subgraph of C constructed 
as follows: 

• Add the output of C to T 

• For every gate v added to T do the following: 

— If v is a +- gate, add exactly one of its children to T. 

— If v is a x-gate, add both of its children to T. 

As C is multiplicatively disjoint, a parse tree of C is a tree. The monomial m(T) 
computed by a parse tree T is the product of the labels of its leaves. The polynomial 
computed by C is the sum of the monomials of all parse trees of C. 
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2.2. Tensors 



In this paper, we interpret tensors as multidimensional arrays. Their algebraic nature 
is not studied here. A good introduction to multilinear algebra and tensors can be 
found in [7]. 

Let n±, . . . , rife be k positive integers. A k- dimensional tensor T of order (m, . . . , n^) 
is a mapping T : [m] x . . . x [rife] — > K. For ii e [ni], . . . ,ik S [rife], we denote by 
r[»i, . . . , ife] the value of the mapping on the point (ii, . . . , ifc). We call these values 
entries of T. We denote by -D(T) the domain of T. Obviously -D(T) = [rc-i] x . . . x [rife]. 

The size of a tensor T, denote by ||T||, is the number of entries, i.e. ||T|| = fli=i n *' 
where T is of order (rii, . . . , rife). The maximal order of T, denote by maxorder(T) is 
max ig[fc] n i- ln t ne following, we also call tensors of dimension 2 matrices and tensors 
of dimension 1 vectors. 

Definition 2 (Contraction). Let T be a k-dimensional tensor of order (n\, . . . ,rife) 
and G an l-dimensional tensor of order (mi, . . . , mi) with k, I > 1. If = mi, we 
denote by T*G £/ie contraction of T and G on the dimensions k and 1 which is a tensor 
of order (m, . . . ,n fc _i,m 2 , . . . , mi) defined as (T*G)[ei, e 2 ] = J27=i T[ei,i]G[i,e 2 ] for 
all ei G [ni] x . . . x [rife_i] and e 2 £ [m 2 ] x . . . x [mj]. 

Remark 3. Obviously, contraction is a direct generalization of the matrix product. 
Indeed, if both T and G are matrices, then T * G is the ordinary matrix product. 

Proposition 4. Let T , G, H be tensors with dim(G) > 2 such that T * (G * H) and 
(T * G) * H are both well defined. Then 

T*{G*n) = {T*G)*H. 

Proof. By direct consequence of the associativity of the ordinary matrix product. For 
a tensor T of dimension k > 2 and order (rz-i, . . . , rife), and a tuple e of length k — 2 we 
define the rti x matrix T e :— (T[i, e, j])ie.[n{\,je[n k ]' Then by associativity of matrix 
product: 

Vei, e 2 , e 3 : T ei * (G 62 * H e3 ) = (T ei * G 62 ) * H e?j . 

Hence, the claim follows when dim(T) > 2 and dim(H) > 2. For dim(T) = 1 or 
dim(_ff) = 1 the argument is similar. □ 

Observation 5. If G is a vector, then the equality of Proposition^ may not be true 
anymore. For example 




Definition 6. A {*}-formula F is a labeled, ordered, rooted binary tree whose the 
leaves, called the inputs, are labeled by tensors whose entries are elements of K. or 
variables and the other nodes are labeled by * . The tensor T v computed by a node v is 
defined inductively: 

• If v is a leaf then T v :— label(w). 
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• If v is labeled by * and has left child V\ and right child vi then T v := F Vl * F V2 . 

A {*} -formula computes the tensor computed by its root. 

As the entries of the input tensors are constants of IK or variables, each entries of 
a tensor computed by a gate is a polynomial of K[-X"i, . . . , X n ], This is why it makes 
sense to compare the computational power of {*}-formulas and arithmetic circuits 
defined in the last section. Moreover, of all the polynomial computed in the output 
of an {*}-formula we will mostly only be interested in one single polynomial. Thus 
we assume that the output tensor has only one single entry, i.e. the tensor is indeed a 
scalar. Observe that this form can always be achieved by contracting with vectors. We 
say that the scalar polynomial computed by a {*}-formula is the polynomial computed 
by it. 

Definition 7. The size of a {*}-formula F, denoted by \F\, is the number of *- gates 
plus the size of the inputs, i.e. \F\ := \{v | label(u) = *}| + J2t-t input of f 11^11- The 
dimension of F, denoted by dim(_F) is the dimension of the tensor computed by F. 
The maximal dimension of F, denoted by maxdim(-F) is the maximal dimension of the 
tensors computed at the gates of F, i.e. maxdim(i 7 ') := max„ : gate in F(dim(T„)). The 
input dimension of F is max„ : „ input of f dim(T„). 

We will often mix the notations for tensors and for tensor formulas. For example, if 
F is a tensor formula computing the tensor T, we will speak of the order of F instead 
of T and write F[e] instead of T[e\. Moreover, given two different formulas F and F', 
we will write F ~ F' if they compute the same tensor. 

3. From arithmetic circuits to {*}-formulas 

We describe how a family of polynomials in VP can be simulated by a family of 
{*}-formulas of polynomial size and maximal dimension 3. Our proof is inspired 
by a proof from [5] where it is shown that polynomials in VP can be represented by 
bounded treewidth CSPs. 

Theorem 8. Let (/„) S VP. There exists a family of {*} -formulas (F n ) of maximal 
dimension 3 and polynomial size such that F n computes f„ for all n. 

We use the following observation from [Bj which can be proved by combining results 
from Malod and Portier j6] and Valiant et al. [llj . 

Proposition 9. Let f be computed by an arithmetic circuit C of size s. Then there 
is an arithmetic circuit C of size s°^ that also computes f such that all parse trees 
of C are isomorphic to a common tree T ■ 

Theorem [5] follows direcly from Proposition [S] and the following lemma: 

Lemma 10. Let C be an arithmetic circuit computing the polynomial f whose parse 
trees are all isomorphic to a common parse tree T. Then there exists a {*} -formula 
F of maximal dimension 3 and of size 9|C| 3 |7"| that computes f. 
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Proof. We construct a tensor formula along the tree T which contains the sum of all 
monomials of /„ in its entries. We denote by V(T) (resp. V(C)) the vertices of T 
(resp. C). For s € V(T^, we call 7^ the subtree of T rooted in s. We define a partial 
parse tree rooted in s to be a function p : V(7^) — > V(C) respecting the following 
conditions for all t £ V(T S )- 

1. If t is a leaf, then is an input of C. 

2. If t has one child t\, p(t) is a +-gate and p{t\) is a child of p{t) in C. 

3. If t has two children t\ and t2, then 

a) p(t) is a x-gate, 

b) p(ti) is the left child of pit), and 

c) pifz) is the right child ofp(t). 

We call these conditions the parse tree conditions. It is easy to see that when s is 
the root of T (and thus T s — T) and p : V(T) — > V(C), then p(V(T)) is the vertex 
set of a parse tree of C if and only if p is a partial parse tree rooted in s. 

If p is a partial parse tree rooted ins S V(7~), we define the monomial m(p) computed 
by P by m(p) := Ilteieaf(r 3 ) label(p(i)). Observe that this is well defined as p respects, 
in particular, the first parse tree condition and thus pit) for t € leaf (7^) is always an 
input of C. If p does not respect the parse tree conditions, we set m(p) = 0. With 
this notation we have 

/ = mip). 

p:V(T)^V(C) 

We index the vertices of C : V(C) = {v\, . . . ,v r } with r — \C\. We denote by E 
the tensor of dimension 1 and order (r) such that for all i < r, E[i] = 1 and by 5ij the 
dirac function which equals 1 if i = j and otherwise. We construct by induction along 
the structure of T a {*}-formula F s for each s G T ■ The formula F s has dimension 
2, order (r, r), size at most 9r 3 |7^| and maximal dimension 3. Furthermore, for all 
i,j < r: 

F s [i,j] = 5ij Y m (p)- 

Observing / = E*F S *E when s is the root of T completes the proof. We now describe 
the inductive construction of F s . Several cases occur: 



s is a leaf: In this case T s consists only of the leaf s. The partial parse trees of T s are 
functions p : {s} — ► V(C) and m(p) = label(p(s)) if p(s) is a input of C and m(p) = 
otherwise. Then F s consists of a r x r input matrix / such that for all i,j < r, 



5ij\abe\ivj) if Vj is an input 
otherwise. 
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Obviously, F s is of size r < 9r 3 , of maximal dimension 2 and 

p:{s}^V(C) 

s has one child si: We start with an observation on functions p : V(7^) — > ^(C). 
Let pi be the restriction of p on V(7^ 1 ). If p is a partial parse tree, then pi is one, 
too, because it fulfills the parse tree conditions for all t € V(T Sl ) Q V(%). Moreover, 
m(p) = m(pi) because the leaves in p and in p\ are the same. In addition, if p is not 
a partial parse tree then 

• either p violates a parse tree condition for t G T Sl . In that case, pi is not a 
partial parse tree and then m(p) = m(pi) = 0, 

• or p(s) is not a +-gate, 

• or p(s) is a +-gate but p(si) is not a child of p(s). 

We encode these conditions in a tensor of dimension 3 and order (r, r, r) defined as 



M[i,j, k] := 
Let F, be the formula 



<5j,fc if is +-gate and V{ is a child of vj 
otherwise. 



F S :=E* (F S1 * M) 

of maximal dimension 3, dimension 2 and order (r, r). We have \F S \ < 9r 3 (|7^| — 1) + 
r 3 +2+||£|| <9r 3 \T s \ and 

r r 

F s \j,k] = j2(E F ^p} M \p>i> k v 

i=l p—1 

r 

= s i,k^2( Yl m{ Pl )M[i,j,j}). 

i=l P1 :V(T S1 )^V(C) 

Let p be a function p : V(T S ) — > V(C) such that p(si) = Vi and p(s) = Vj. Let p\ 
be its restriction on V(T Sl ). We have m(p) = m(pi)M[i,j,j], because 

• if p is a partial parse tree then M[i 7 j,j] = 1 and thus m( P i)M[i,j,j] = m( P i) = 
m(p), 

• if Vj is not a +-gatc then m(p) ~ and also m(p 1 )M[i 1 j, j] = because 
M[i,j,j] = 0, 

• if pi(si) = Vi is not a child of vj = p(s) then m(p) = 0. Since M[i 7 j,j] — 0, we 
have m(p) = = m(pi)M[i,j,j}. 



7 



Thus we have in each case 

r 

F s [j,k] = Sj ik ^2 ^2 m(Pi) M [ i J>k]=6 jtk m(p) 

j = l P1 :V(T S1 )^V(C) p:V(T s )^V(C) 
p 1 (s 1 )=v i vM = Vj 

which completes the proof of this case. 

s has two children, s\ (left child) and s 2 (right child): As above, we encode 
the parse tree conditions in tensors of dimension 3 and contract them correctly to 
compute the desired result. This time there are two different tensors: one encoding 
the condition 3.b and one for 3.c. Let Ml and M R be the two following (r, r, r) tensors: 

8j t k if Vj is a x-gate and Vi is the left child of vj 



Mr \i 7 fcl - / Sj ' k ^ Vj ^ ' 
M L [%,j,k\-^ Ootherwise; 



„ ,. . „ f Si h if Vi is a x-gate and v; is the right child of u,- 
M R \%, 3 ,k] = [ o otherwise. 

Let F s be the formula of maximal dimension 3, dimension 2 and order (r, r) defined 

as 

F S = (E* (F S1 * Ml)) * (E * (F S2 * M fl )). 
We have |F S | = \F Sl \ + \\M L \\ + \\M R \\ + \F S2 \ + 5 + 2\\E\\ < 9r 3 \%\. In addition: 

r r r 

F a [i,j] = ^((£ i F Sl [a,a]ML[a,i,k]) x (^F S2 [b,b]M R [b,k,j])) 

k=l a=l b=l 

r 

= Si,j F Sl [a,a}M L [a,i,i}F S2 [b,b}M R [b,i,i] 



a,b=l 



= Sj,j ^ y~] ^2 m(pi)m(p2)M L [a,i,i]M R [b,i,i] 

a ,b=l Pi:V(T sl )-»V(C) p 2 :V(T 32 )^V(C) 
Pl("l)=«a P2(«2) = «6 

Similarly to before consider p : V(T S ) — > V(C) such that p(s) — Vi, p(si) = v a and 
p{s2) = Vb- We denote by p\ the restriction of p on V(T Sl ) and by pi its restriction on 
V(T S2 ). We will show 

m(p) = M L [a,i,i}M R [b,i,i]m(p 1 )m(p2) 
by studying the possible cases: 

• If p is a partial parse tree then p\ and P2 are, too. Moreover, since s has two 
children, p(s) = Vi is necessarily a x-gate, v a its left child and Vb its right child. 
It follows that Mi [a, i, i] — M R [b, = 1 and 

m(p) = f] label(Z) = JJ label(Z) ]~| label(Z) = m(p 1 )m(p 2 ). 

iGleaf(T 3 ) ;eleaf(T ai ) ieleaf(T S2 ) 
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• If p is not a partial parse tree then three cases can occur: If p\ (resp. p%) is 
not a partial parse tree, then m{p\) = (rcsp. m{p2) = 0). If Vi is not a 
x-gate, then Mi[a,i,i] = 0. Finally, if v a (resp. v ) is not the left (resp. 
right) child of Vi, then Ml[cl, = (resp. Mji[b, = 0). In all those cases, 
M L [a,i,i]M R [b,i,i]m(pi)m(p 2 ) = = m(p). 

This completes the proof. □ 



4. From {*}-formulas to arithmetic circuits 

In this section wc will show that the polynomials computed by polynomial size {*}-formulas 
can also be computed by polynomial size arithmetic circuits. We start by first proving 
this for formulas with bounded maximal dimension. Then we extend this result by 
showing that any {*}-formula can be transformed into an equivalent one with bounded 
maximal dimension without increasing the size. 

4.1. Formulas with bounded maximal dimension 

Proposition 11. Let F be a {*}-formula of maximal dimension k, dimension I < k 
and order (m, . . . , n{). Let n := maxr-.T input of F(maxorder(T)). Then there exists a 
multiplicatively disjoint circuit C of size at most 2n fe+1 |-F| such that for all e G D(F) 
there exists a gate v e in C computing F[e]. 

Proof. If F is an input, let C be the circuit having Y\\ =1 n,- L inputs, each one labeled 
with an entry of F. The size of C is ]X=i n i — nk ■ 

If F = G * H, by induction we have circuits Cq and Ch with the desired prop- 
erties for G and H. The dimension of F is less than k and for e G D(F), F[e] = 
HT=i G[ei,i]H[i,e 2 ] with m < n. 

Each G[ei,i] and H[i,e2] is computed by a gate of G'g and Ch, respectively, so we 
can compute F[e] by adding at most 2n gates (to x-gates and to — 1 +-gates). As 
there are at most n k entries in F, wc can compute all of them with a circuit C by 
adding at most 2n x n k gates to Ch U Cg ■ 

The circuit C is multiplicatively disjoint since each x-gate receives one of its input 
from C G and the other one from C H - Also |C| = \C G \ + \C H \ + 2n k+1 < 2n fe+1 |F|. □ 

Corollary 12. Let (F n ) be a family of {*} -formulas of polynomial size and of maximal 
dimension k computing a family (/„) of polynomials. Then (/„) is in VP. 

4.2. Unbounded maximal dimension 

Since the size of the circuit constructed in the previous section is exponential in k 
maxdim(i 7 '), we cannot apply the results from there directly if k is not bounded by 
a constant. Somewhat surprisingly we will see in this section that one does not gain 
any expressivity by letting intermediate dimensions of formulas grow arbitrarily. Thus 
bounding maxdim^) is not a restriction of the computational power of {*}-formulas. 
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Definition 13. A {*}-formula F of dimension k and input dimension p is said to be 
tame if rnaxdmi(F) < max(fc,p). 

Definition 14. A {*}-formula F is said to be totally tame if each subformula of F 
is tame. 

Let us remark again that also in [5] and [5] there is a notion of tameness that prevents 
intermediate results from growing too much during the computation. It turns out 
that in those papers tameness plays a crucial role: Tame formulas can be evaluated 
efficiently while general formulas are hard to evaluate in the respective models. We 
will see that in our setting tameness is not a restriction at all. Indeed, any {*}-formula 
can be turned into an equivalent totally tame formula without any increase of its size. 
Thus totally tame and general formulas have the same expressive power in our setting 
which is a striking difference to the setting from [5] and [3] . We start with the following 
lemma: 

Lemma 15. Let F be a totally tame formula with dim(_F) = k and input dimension p. 
For all totally tame formulas E of dimension 1 and input dimension at most p, there 
exist totally tame formulas G r and Gi of size \F * E\ = \E * F\ = \F\ + \E\ + 1 such 
that G r ~F*EandGi~E*F. 

Proof. We only show the construction of G r ; the construction of Gi is completely 
analogous. We proceed by induction on F . 

If F is an input, then maxdim(i ? ) = dim(i 7 ') = p. Let E be any totally tame formula 
of dimension 1 and input dimension at most p. We set G r := F * E. Clearly, k = 
dim(G r ) =p— 1. Furthermore, maxdim(GV) = max(p — 1, maxdim(i 7 '), maxdim(-E)) < 
p because E has input dimension at most p and is totally tame. Thus G r is totally 
tame. 

Let now F — F\ * F 2 . Let ki := dim(Fi) and k 2 ■— dim(F 2 ). Let £ be a totally 
tame formula of dimension 1 and input dimension at most p. 

• If dim(F2) = 1, we claim that G r = F * E is totally tame. Indeed, since 
dim(i 7 2) = 1, we have dim(i^) = k\ — 1. But F is by assumption tame, so 
k\ = dim(-Fi) < max(fci — Hence k\ < p and dim(F) < p. Thus all 
intermediate results of G r have dimension at most p, so G r is tame. But then it 
is also totally tame, because its subformulas are totally tame by assumption. 

• If dim(i 7 2) = 2, we have p > 2 obviously and dim(i 7 ') = dim(i 7 'i). F 2 is a 
subformula of F, so it is totally tame, too. Furthermore, F 2 * E is of dimension 
1 and it is also totally tame since 2 < p. Moreover, by Proposition 0] we have 
F * E ~ Fi * (F 2 * E). Applying the induction hypothesis on F\ and (F 2 * E) 
gives the desired G r . 

• If &m\(F 2 ) > 2, by Proposition 2] we have F * E ~ F\ * (F 2 * E). We first apply 
the induction hypothesis on F 2 and E to construct a totally tame formula G' 
computing F 2 * E. Finally G r := F\ * G' is totally tame since Fi and G' are 
totally tame and F is of dimension k = k\ + k 2 — 2 > max(/ci, k 2 — 1). 



10 



□ 



We now prove the main proposition of this section: 

Proposition 16. For every {*}-formula F there exists a totally tame {*}-formula F' 
such that F' ~ F and \F\ = \F'\. 

Proof. Proof by induction on F. 

If F is an input then it is trivially totally tame as the dimension of F is equal to 
the input dimension of F. So we set F' := F. 

If F = F\ * F2 then several cases can occur depending on the dimension of F 1 and 
F 2 . We denote by k, k\, k 2 the dimensions of F, F\ and F2 respectively. We recall that 
k = fci + k 2 - 2. 

• If both ki and k 2 are different from 1. Then F 1 = F{ * F 2 is totally tame since 
k > max(/ci, k 2 ) 

• If k% = 1 or k 2 = 1, we use Lemma [15] on F[ and F 2 to construct F' of size \F\, 
totally tame, computing F\ * F 2 . 

□ 

Combining Proposition [T51 and Corollary rj2] we get the following theorem: 

Theorem 17. Let (F n ) be a family of {*} -formulas of polynomial size and input 
dimension p (independent of n) computing a family of polynomials (f n )- Then (f n ) is 
in VP. 

Proof. Applying Proposition \W\ on (F n ) gives a family (F^) computing (/„) such that 
(i 7 ^) is tame. Then the maximal dimension of F' n is p (because F n is scalar, thus of 
dimension 1) and applying Corollary 1121 proves the claim. □ 

4.3. Unbounded input dimension 

While we got rid of the restriction on the maximum dimension of {*}-formulas in the 
last section, we still have a bound on the dimension of the inputs in Theorem 1171 In 
this section we will show that this bound is not necessary to have containment of 
the computed polynomials in VP. We will show that inputs having "big" dimension 
can be computed by polynomial size {*}-formulas of input dimension 3. We can then 
use this to eliminate inputs of dimension more than 3 in {*}-formulas. Applying 
Theorem [T7] we conclude that the only restriction on {*}-formulas that we need to 
ensure containment in VP is the polynomial size bound. 

Proposition 18. Let T be a r-dimensional tensor of order (n\, . . . ,n r ). Let L := 
\\T\\ = nj_i n i be the number of entries in T . Then there is a {*}-formula F of size 
r + 1 + L 3 + 2L and input dimension 3 computing T. 
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Proof (sketch). Choose an arbitrary bijection B : [L] — > [m] x . . . x [n r ]. Let further- 
more Bi : [L] — > [rii] for i < r be the projection of B onto the i-th coordinate. We 
define the 3-dimensional tensors Tj of order (L,m,L) by 



Ti[m, k, n] 
and, for 2 < i < r, 



T[B(m)\ if m — n and Z?i(m) = fc 
otherwise. 



_ , , I 1 if m = n and BAm) = k 

li\m, k, n \ = < _ ,, 
n ' ' 1 otherwise. 



By induction one can show that for the tensor P = T\ * . . . * T r we have that 



P[m, hi, . ■ ■ , k r , n] 



T[ki, . . . ,k r ] if m = n and B(m) — (fci, . . . , k r 
otherwise. 



Hence T = E * P * E where E is a vector of order L filled with 1. The complete 
proof is given in the appendix. □ 

The following theorem is a direct consequence of Proposition [T5] and Theorem IT71 

Theorem 19. Let (F n ) be a family of {*}- formulas of polynomial size computing a 
family of polynomials (/„). Then f n is in VP. 



5. The power of contracting with vectors 

In this section we will make a finer examination of where exactly the additional power 
originates when going from iterated matrix product of [B] to tensor contractions. We 
will see that this additional expressivity crucially depends on the possiblity of con- 
tracting tensors on more than two of their dimensions. We will show that when we 
prevent this possibility by disallowing contractions with vectors - which are used in 
the proof of Theorem [5] to "collapse" dimensions not needed anymore so that we can 
access other dimensions to contract on - the expressivity of {*}-formulas drops to that 
of iterated matrix product. 

Observe that we cannot assume that {*}-formulas compute scalars in this setting, 
because we cannot decrease the dimension of the tensors computed by a formula. Also 
we cannot compute all entries of the output at the same time efficiently, because those 
might be exponentially many ones. But we will see in the following Propositions that 
we can compute each individual entry of the output more efficiently than in the general 
setting where contraction with tensors is allowed. 

Proposition 20. Let F be a {*} -formula of order (rii, . . . ,rik) whose inputs are all 
of dimension at least 2. Then for all e £ D(F) there exists a skew arithmetic circuit 
C of size at most 2n 3 |F| where n := maxr : t input of F(maxorder(T)) computing F[&]. 
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Proof. By Proposition [4] we can write F as A\ * {A.% * {A3 * ... * A n )). The proof then 
follows easily by induction: We do the same construction as in Theorem [12] but this 
time we only have n 2 entries and at each *-gate, one side is an input, resulting in a 
skew circuit. □ 

The case of Proposition I2TJ1 exactly corresponds to the characterization of VP WS by 
Malod and Portier [B] by n products of matrices of size n x n. Thus Proposition [30] 
naturally generalizes this result and the real new power seen in Theorem [5] must come 
from the use of vectors in the products. As we have seen in the proof of Propositionl20lit 
is crucial that vectors are the only case which breaks the associativity of Proposition 2J 
So what looked like a not very important edge case in Observation[S]plays a surprisingly 
important role for the expressivity of {*}-formulas. 



6. The *y operators 

Our characterization of VP by {*}-formulas contracts on dimension only in a very 
specific way in the contraction of two tensors: We always only contract on the last 
dimension of one tensor and the first dimension of the other one. It is thus very natural 
to ask if this is a restriction of the computational power of the formulas. In this section 
we will see that it is indeed not. If we allow free choice of the dimensions to contract on 
during a contraction this does not make the resulting polynomials harder to compute. 
To formalize this we give the folowing definition of a contraction 

Definition 21. Let T be a k- dimensional tensor of order (ni, . . . , rif.) and G a l- 
dimensional tensor of order (mi, . . . , TO;) with k, I > 1. When Hi = m,j for i < k and 
j < I, we denote by T *ij G the contraction ofT and G on the dimensions i and j the 
(k+l— 2) -dimensional tensor of order (n\, . . . , rij_i, Uj+i, . . . , n^, mi, . . . , rrij—i, m 3+ i, . . 
defined as 

rii 

( T *hj G)[ei,e 2 ,e 3 ,e 4 ] = T[e 1} r, e 2 ]G[e 3 , r, e 4 ] 

for all ei G [n\\ x . . . x [rii_i], e 2 £ x . . . x [nk], e 3 £ [mi] x . . . x [m.j_i] and 

e 4 G [mj+i] x ... x [mi]. 

{*jj}-formulas are defined in complete analogy to {*} -formulas. 

It turns out that {*ij}-formulas cannot compute more than {*}-formulas, so the 
free choice of the dimensions to meld on does not change much. 

Theorem 22. Let (F n ) be a family of -formulas of polynomial size computing a 
family of polynomials (/„). Then (/„) is in VP. 

The proof of Theorcm[22]follows a similar approach as that of TheoremQlJland is thus 
given in the appendix for lack of space. Let us sketch some key steps here: If we bound 
the maximal dimension of {*i.j}-formulas by a constant k, it is easy to see that the 
proof of Theorem [T2l can be adapted to {*ij}-formulas in a straightforward way. The 
main complication is then turning general {*i j}-formulas into totally tame ones. *jj 
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is not associative anymore, and this makes a straightforward translation of the proof 
of Proposition [16] tricky. These problems can be solved by the observation that the 
crucial steps in the process of making a formula tame are those where a {*,.j}-formula 
is multiplied by a tensor of dimension 1. But for such contractions we can give ex- 
plicit formulas for different cases that may occur, so again every {*jj}-formula has 
an equivalent tame {*i.j}-formula. Combining this with Proposition IT51 completes the 
proof. 

7. Conclusion 

We have shown that one can get a robust characterization of VP by formulas with 
tensors as input and tensor contraction as the only operation. This generalizes the 
known characterization of VP WS by iterated matrix product by Malod and Portier [B] . 
In some aspects the situation in our setting is more subtle, though. We remarked 
that vectors and in general breaking associativity plays a crucial role if we want to 
characterize VP. Also, unlike for iterated matrix product we have to make a choice 
if we take *.y or * as our basic operation. It is easy to check that that using the 
equivalent to *jj for matrix product would merely be transposing the matrix, so it 
clearly does not change the expressivity of the model. But fortunately also in our 
setting, the choice of *jj or * does not influence the complexity of the computed 
polynomials. 

Unfortunately, unlike for iterated matrix product our characterization seemingly 
does not directly lead to a characterization of VP by something similar to branching 
programs. We still think that such a characterization is highly desirable, because the 
branching program characterization of VP WS has been the source of important insights 
in arithmetic circuit complexity. Thus we believe that a similar characterization of 
VP might lead to a better understanding of VP, a class that is arguably not very well 
understood, yet. 

Let us quickly discuss several extensions to the results in this paper that we had 
to leave out for lack of space: First, analyzing the proofs of Section [U a little more 
carefully one can see that our results remain true if one does not measure the size of 
a tensor as the number of its entries but as the number of its nonzero entries. This 
makes it possible to allow inputs of large dimension and large order. 

Also, it seems plausible and straightforward to generalize our results to arbitrary 
semi- rings in the style of Damm, Holzer and McKenzie [2]. Choosing different semi- 
rings one would then probably get characterizations of classes like LOGCFL and its 
counting, mod-counting and gap-versions. The main new consideration would be the 
treatment of uniformity in these settings which appears to be possible with a more 
refined analysis of our proofs. 

Finally, for tensors there are other natural operations to perform on them like addi- 
tion or tensor product. It is natural to ask, if adding such operations does change the 
complexity of the resulting polynomials. While it is straightforward to see that adding 
only tensor product as an operation does not increase the expressivity of {*}-formulas, 
we could so far not answer the corresponding question for addition. Therefore, we 
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leave this as an open question. 
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A. Proof of proposition [TBI 



Choose an arbitrary bijection B : [L] — ► [ni] x . . . x [n r ]. Let furthermore Bi : [L] — > [nj 
for i < r be the projection of B onto the i-th coordinate. 
We define the 3-dimensional tensors Ti of order (L,rii,L) by 



Ti[m, k, n] 

and 



T[B{m)\ if to = n and Bi{m) = k 
otherwise. 

1 if m = n and Bi(m) = k 



, , f 1 if m = » a 
n ' ' J 10 otherwise. 



for i > 1. 

We prove by induction that for the tensor Pj := T\ * . . . * Tj, j < r, we have 
that Pj [to, h\, . . . , kj, n] = T[k\, . . . , k r ] if to = n and B(m) = and 
Pj[m, k\, . . . ,kj,n] =0 otherwise. 

For j = 1 is obvious by definition of T\ . So assume that it is true for j — 1 then 

Pj[m, fei,..., fcj,n] = X)p=i p j-i[ m > fc i>---> fc j-i>-P] T i[P ; fc ^ n ] 
= Jj-i [m, fci, . . . , , m]Tj [to, fcj, n] 

T[_B(m)] \i m — n and Bi(m) — ki for i < j — 1 

and Bj(m) = kj 
otherwise. 

This concludes the induction. 

Thus we have P r [m, fci, . . . , fc r , n] = T[fci, . . . , k r ] if B(m) = {k\, . . . , fc r ) and to = n. 
We now sum over all n, to G [L] by F = E * P r * E where E is a vector of size L 
containing only Is. The resulting {*}-formula is of size r + 1 + L 3 + 2L because each 
Ti is of size L 2 rii < L 3 and E is of size L and we use r + 1 *-gates. 



B. Proof of Theorem 22 



The proof of Theorem [55] follows a similar approach as that of Theorem \W\ Let us 
first observe that an analogous version of Theorem [T2] with *jj instead of * can be 
proved easily. Also Proposition [TBI applies directly for 

Thus the only thing left to prove is that every {*ij}-formula can be turned into a 
totally tame one. *ij is not associative which makes a straightforward translation of 
the proof of Proposition [Tf)] tricky. Still it is possible observing that for the crucial 
case of contraction by a vector, it is possible to prove identities that we can use where 
we applied associativity before. 
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Proposition 23. Let F\ and F 2 be two tensors of dimension k\ and k 2 , respectively. 
Let E be a tensor of dimension 1. Then 

{Fx *ij (F 2 E) ifki+j-l<k 

Fx (F 2 *fc-fc 1+ x,i E) if fci < k < fci + j - 2 

(F 1 * ka E) H , j F 2 ifi<k<k x 

{F\ *kA E) F% ifl<k<i 

Proof. The proof follows from simple calculation. We denote by (tlx, . . . , Wfe-J and 
(mi , . . . , m,t 2 ) the orders of Fx and F 2 , respectively. Let F be the product F = F\ *ij F 2 ■ 
We have 

rii 

F[ei,e 2 ,e 3 ,e 4 ] = F x [ei, r, e 2 }F 2 [e 3 , r, e 4 ] 

r=l 

where ei is a tuple of length i— 1, e 2 of length fci — i, e 3 of length j — 1 and e 4 a tuple 
of length k 2 — j. 

If fci + j - 1 < fc, then 

m 

(F * E) [ei , ea, e a , f , f] = ^ F[ei, e 2 , e 3j f , p, f']£[p] 

= £ X] ( Fl ^ ' r ' ^ ^3 » r ' f ' f '] ) E W 

p— 1 r— 1 

= E E F i[ e i' r ' e 2](^2[e 3 ,r, f,p, f']^[p]) 

r— 1 p— 1 

= i*i (-Fa *fc-fci+i,i E) 

where f , f ' are the suitable subtuples of e 4 . The other cases can be checked in the 
same way. The only difference is the position of f and f. □ 

Using these identities, we prove the following lemma. 

Lemma 24. Let F be a totally tame -formula of dimension k and input dimen- 
sion p. For all totally tame formula E of dimension 1 and for all i < k, there exists a 
totally tame {*ij}-formula G of size \F *o E\ such that G ~ F *^i E. 

Proof. The proof is done by induction on F. 

If F is an input, let E be any totally tame formula of dimension 1 and i < k. Let 
G = F *j ) i E. Then G is obviously totally tame since F and E are totally tame and 
G is of dimension p — 1 so all the intermediate tensors are of dimension at most p. 

If F = Fx *ij F 2 , then several cases can occur. Let E be any totally tame formula 
of dimension 1 and I < k. We denote by fci and k 2 the dimensions of Fx and F 2 
respectively. We want to compute F *i t x E. We proceed differently depending on 

If fci < I, then we know, using the identities from Propositionl23lthat either F*i y iE ~ 
Fi {F 2 *i-ki+i,i E) or F * u E ~ Fx {E 2 *i-k 1 +i,i E). So in both cases, there 

exists j' such that F *;.i E ~ Fx (F 2 *i-ki+i,i E). Depending on the dimension of 
i*2) we again consider different cases: 
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• First of all, the case dim(i<2) = 1 is not possible because if fc 2 = 1 then k = k\ — 1 
and then necessarely I < fci 

• If dim(i<2) = 2 then F 2 *| fc x +i,i £ is a formula of dimension 1. Moreover, 

it is totally tame because F 2 is totally tame since F is, E is totally tame by 
assumption and dim(F 2 *i-k 1 +i,i E) = 1 < p. Thus we can apply the induction 
hypothesis on F\ and the vector F2 *i-k 1 +i.i E 

• If dim(i 7 2) > 2 then we first construct G by applying the induction on F2 and 
E. The formula G = F\ *ij> G' computes F and is totally tame because both F\ 
and G 1 are totally tame and that dim(F) = k\ + fc 2 — 2 > max(fci, fc 2 — 1) since 
k 2 > 2 

This completes the case fci < I. 

We proceed similarly for the case I < k\ using the other identities from Poposition 
[231 In this case we have that there exists i' such that F E ~ (F\ E) *i>j F2. 
Again, we analyse depending on the dimension of F\ . 

• Again, since I < fci, we have dim(Fi) ^ 1. 

• If dim(Fi) = 2 then (Fi E) is a formula of dimension 1. So i' = 1 and thus 
F ~ F2 (Fi *i t i E). As before, F\ *i t \ E is totally tame and we apply the 
induction on F 2 and Fi*i^E 

• If dim(i 7 'i) > 2 then first construct G' by applying the induction on F\ and B. 
The formula G — G' F 2 computes F and is totally tame because both F 2 
and G 1 are totally tame and that dim(F) = k\ + fc 2 — 2 > max(Ai, fc 2 — 1) since 
fc 2 > 2. 

□ 

We now prove that {*,;.j}-formulas can always be turned into equivalent totally tame 
{*ij}-formulas. 

Proposition 25. Let F be an -formula. Then there exists a totally tame -formula F' 
such that F' ~ F and \F\ = 

Proof. The proof is done by straightforward induction on F. 

If F is an input then it is trivially totally tame as the dimension of F is equal to 
the input dimension of F. We simply set F' := F. 

If F = F% *ij F2 then two cases can occur depending on the dimension of F\ and F 2 . 
We denote by k, fci,fc 2 the dimensions of F, F\ and F 2 , respectively. We recall that 
k = ki + fc 2 - 2. 

• If both fci and fc 2 are greater than 1, then F' = F[ *jj F' 2 is totally tame since 
k > max(fci, fc 2 ). 

• If fc 2 = 1 or fci = 1, we use Lemma [24] on F[ and F' 2 to construct F' of size 
totally tame, computing F\ * F%. 
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□ 

This completes the proof of Theorem 
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